Administering Defining Issues Test Online: Do Response Modes Matter?

The purpose of the study was to determine comparability of an online version to the original paper-pencil version of Defining Issues Test 2 (DIT2). This study employed methods from both Classical Test Theory (CTT) and Item Response Theory (IRT). Findings from CTT analyses supported the reliability and discriminant validity of both versions. Findings from IRT analyses confirmed that both versions had comparable power of test-of-fit to the Rasch model. However, IRT analyses found that there were some variations in item difficulties and the patterns of item functions between the two versions. The study also examined students’ satisfaction of DIT2taking experience in the two survey response modes. The ANOVA results indicated that the online version of DIT2 was comparable to the paper-pencil version in terms of ease of use.